Conditional Value-at-Risk Minimization in Finite State Markov Decision Processes: Continuity and Compactness

This study is concerned with the dynamic risk-analysis for finite state Markov decision processes. As a measure of risk, we consider conditional value-at-risk(CVaR) for the real value of the discounted total reward from a policy, under whose criterion risk optimal or deterministic policies are defined. The risk problem is equivalently redefined as a non-linear optimization problem on the attainable set of the distribution functions for the real values over all policies. Showing the weak-continuity of CVaR on the space of attainable distribution functions, the mathematical existence theorem of optimal policies are proved throughout the discussion of convex analysis and weak-compactness. c ©2013 World Academic Press, UK. All rights reserved.